Histology recognition to automatically score and quantify cancer grades and individual user digital whole histological imaging device

ABSTRACT

Digital pathology is the concept of capturing digital images from glass microscope slides in order to record, visualize, analyze, manage, report, share and diagnose pathology specimens. The present disclosure is directed to a desktop slide scanner, which enables pathologists to scan slides at a touch of a button. Included is a workflow for reliable imaging, diagnosis, quantification, management, and sharing of a digital pathology library. Also disclosed herein is an analysis framework that provides for pattern recognition of biological samples represented as digital images to automatically quantitatively score normal cell parameters against disease state parameters. The framework provides a pathologist with an opportunity to see what the algorithm is scoring, and simply agree, or edit the result. This framework offers a new tool to enhance the precision of the current standard of care.

STATEMENT REGARDING FEDERALLY FUNDED RESEARCH

This invention was made with government support under Grant No.U54CA143970 awarded by the National Institute of Health. The governmenthas certain rights in the invention.

BACKGROUND

Pathology is the specialized scientific study of the nature of diseaseand the medical practice of diagnosis of those diseases. Disease isoften manifested in anatomical changes which can be visually determined.Pathologists, whether surgical, cytopathological, forensic, veterinaryor other, view these changes from a normal state and make a diagnosis ofthe disease state. For example, cancer is diagnosed by visuallyidentifying the structural alteration of cells. In order to simplify anddescribe the disease, many common cancer types are graded based on theseverity or aggressive phenotype of the disease. The amount ofphenotypic change from normal is described in multiple grading systemsincluding Gleason (prostate), Nottingham (breast), Fuhrman (kidney),etc.

Each of these specialized grades includes a small number of visuallyidentifiable criteria on the hematoxylin and eosin stained microscopeslide. Several tumor types are simply graded I-IV, and are basedcellular differentiation (i.e., how different the cells look compared tonormal). Other factors that may be considered, depending on the tumortype, are structural formation and cellular proliferation (growth rate).The histological grade often has a profound influence on clinicaldecision making. This practice has been traced back to von Hansemann inthe 1890s. Many of the grading systems, such as the Nottingham Score forbreast cancer, are as simple as three visual clues and have remainedwholly unchanged since the 1920s.

Unfortunately, discrepancies have arisen between groups of pathologistson a series of equivocal cases. This is due in large part to thechallenging estimation and judgment calls which need to be made understress. For example, pathologists may use a number of properties indeciding the nature of a cell. These properties often do not have arigid definition. Thus, a pathologist provides a pathological decisionbased on the pathologist's particular experience.

However, with the advent of digital histological slide scanning (1999),massively powerful computational power and robust and reliablealgorithms, novel methods are being sought to grade disease states andmore precisely grade many common cancer types.

Digital pathology takes advantage of high computing efficiency and largevolumes of available computational storage to create digital images ofglass microscopy slides enabling a virtual microscopy to outfit thepathologist's toolbox. Automated slide scanners provide pathologydepartments with the high throughput tools necessary to capture tens ofthousands of whole slide images every year. Slide scanners automaterepeatable imaging conditions for whole slides, which enables a clinicto digitally image slides and make these digital images available to allpathologists within each network system.

While high-throughput slide scanners area generally available forpathology laboratories, desktop single slide scanners are not common.Desktop scanners would enable the individual pathologist to scan slidesat a touch of a button and could be integrated with specialized softwaretools to establish a one-stop workflow for reliable imaging, diagnosis,quantification, management, and sharing of their own digital pathologylibrary.

SUMMARY

The present disclosure describes systems and methods that use theAmerican Joint Commission on Cancer (AJCC) and the College of AmericanPathology (CAP) guidelines for qualitatively grading cancer types andtranslates the rules of visual identification to computationalalgorithms capable of accurately and consistently grading cancer. Thesystems and methods may aid physicians in their decision making. Byoffering a companion diagnostic algorithm to pathologists, the physiciannow has an additional tool in their arsenal to confirm, document andreport potentially diagnostic data.

In accordance with the above, there is provided a computer-implementedanalysis framework that provides for pattern recognition of biologicalsamples represented as digital images to automatically quantitativelyscore normal cell parameters against disease state parameters in thesame way the governing bodies of the AJCC and CAP recommend pathologistsperform the same duties. The framework provides a pathologist with anopportunity to see what the algorithm is scoring, and simply agree, oredit the result. This framework offers a new tool to enhance theprecision of the current standard of care.

In accordance with some implementations, there is provided a desktopscanning device that may be used to in conjunction with a personalcomputer. The desktop scanning device may be roughly the size of astandard 3-ring binder. The desktop scanning device may be used bypathologists to digitally scan, share, analyze, report or otherwisedocument the slides they are currently viewing.

This summary is provided to introduce a selection of concepts in asimplified form that are further described below in the detaileddescription. This summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

Many aspects of the disclosure can be better understood with referenceto the following drawings. The components in the drawings are notnecessarily to scale, emphasis instead being placed upon clearlyillustrating the principles of the present disclosure. Moreover, in thedrawings, like reference numerals designate corresponding partsthroughout the several views:

FIG. 1 illustrates an exemplary environment;

FIG. 2 is a high level operation flow diagram illustrating a method forautomated biological sample analysis;

FIG. 3A illustrates an operational flow diagram that illustrates amethod for the determination of features of the biological samplerepresented by digital images and grading such features;

FIG. 3B illustrates an operational flow diagram of example processesthat are performed at 304 in FIG. 3A;

FIGS. 4A and 4B illustrate several representative graded digital imagesof biological samples;

FIGS. 5A-5C illustrate an desktop slide scanning device;

FIG. 6 illustrates an exemplary environment in which the desktop slidescanning device may operate;

FIG. 7 illustrates an operational flow diagram that illustrates a methodfor scanning slides using the desktop slide scanning device of FIGS.5A-5C; and

FIG. 8 shows an example computing environment.

DETAILED DESCRIPTION

Digital pathology is the concept of capturing digital images from glassmicroscope slides in order to record, visualize, analyze, manage,report, share and diagnose pathology specimens. This practice is beingintegrated in pathology departments to increase productivity, workflowefficiency and the ability to quantify results. In particular, slidescanners automate repeatable imaging conditions for whole slides. Aswill be described herein below, the present disclosure providesimplementations of a desktop single slide scanner that, e.g., willenable pathologists to scan slides at a touch of a button. Furthermore,when integrated with specialized software tools, pathologists mayestablish a one-stop workflow for reliable imaging, diagnosis,quantification, management, and sharing of their own digital pathologylibrary.

FIG. 1 is a block diagram illustrating an exemplary automated digitalimage based biological sample feature detection and classificationsystem 100. The system 100 may include one or more computers 102 with acomputer display 104 (only one of which is illustrated). The computerdisplay 104 may present a graphical user interface (“GUI”) 106 to auser. The computer display 104 may provide for touch-based manipulationof information, images and other user interfaces. The system 100 mayoptionally include a microscope or other magnifying device (notillustrated). The system 100 further includes a digital or analog camera108 used to provide plural images 110 in various image or data formats,as will be described in other implementations with reference to FIG. 6,a slide scanner 500 may be used in conjunction with the camera 108.

One or more databases 112 may store biological sample information asdigital images or in other digital data formats. The one or moredatabases 112 may also include raw and/or processed digital images andmay further include knowledge databases created from automated analysisof the digital images 110. For example, the databases 112 may includevoice annotation of records. The one or more databases 112 may beintegral to a memory system on the computer 102 or in secondary storagesuch as a hard disk, solid state media, optical disk, or othernon-volatile mass storage devices. The computer 102 and the databases112 may also be connected to an accessible via one or morecommunications networks 114 and/or distributed across componentsconnected to the communications networks 114. All memory systems andcomputer-readable media disclosed herein are intended to be tangiblememory systems.

In the above system 100, the one or more computers 102 include, but arenot limited to desktop computers, laptop/notebook computers,workstations, thin clients, mobile devices, tablet computers, smartphones, personal digital assistants (PDA), Internet appliances, etc. Anexample computing device is shown in FIG. 8.

The communications network 114 may include, but is not limited to, theInternet, an intranet, a wired or wireless Local Area Network (LAN orWiLAN), a Wide Area Network (WAN), a Metropolitan Area Network (MAN),Public Switched Telephone Network (PSTN) and other types ofcommunications networks 114. The communications network 114 may includeone or more gateways, routers, or bridges. The communications network114 may include one or more servers and one or more web-sites accessibleby users to send and receive information useable by the one or morecomputers 102. The communications network 114 includes, but is notlimited to, data networks using the Transmission Control Protocol (TCP),User Datagram Protocol (UDP), Internet Protocol (IP) and other dataprotocols.

The one or more databases 112 may include plural digital images 110 ofbiological samples taken with a camera such as a digital camera andstored in a variety of digital image formats including, but not limitedto, TIFF (without compression). However, the present disclosure is notlimited to these digital image formats and other digital image ordigital data formats can also be used to practice the subject matter ofthe disclosure. The digital images 110 are typically obtained bymagnifying the biological samples with a microscope or other magnifyingdevice and capturing a digital image of the magnified biological sample.

Each digital image 110 typically includes an array, usually arectangular matrix, of pixels. Each “pixel” is one picture element andis a digital quantity that is a value that represents some property ofthe image at a location in the array corresponding to a particularlocation in the image. Typically, in continuous tone black and whiteimages the pixel values represent a gray scale value. Pixel values for adigital image 110 typically conform to a specified range. For example,each array element may be one byte (i.e., eight bits). With one-bytepixels, pixel values range from zero to 255. In a gray scale image a 255may represent absolute white and zero total black (or visa-versa). Colordigital images consist of three color planes, generally corresponding tored, green, and blue (RGB). For a particular pixel, there is one valuefor each of these color planes, (i.e., a value representing the redcomponent, a value representing the green component, and a valuerepresenting the blue component). By varying the intensity of thesethree components, all colors in the color spectrum typically may becreated.

Data may be maintained on a tangible computer readable medium includingmagnetic disks, solid state media, optical disks, organic memory, andany other volatile (e.g., Random Access Memory (“RAM”)) or non-volatile(e.g., Read-Only Memory (“ROM”), flash memory, etc.) mass storage systemreadable by the CPU. The computer readable medium includes cooperatingor interconnected computer readable medium, which exist exclusively onthe processing system or can be distributed among multipleinterconnected processing systems that may be local or remote to theprocessing system.

Term Definitions

As used herein, the term “sample” includes cellular material derivedfrom a biological organism. Such samples include but are not limited tohair, skin samples, tissue samples, cultured cells, cultured cell media,and biological fluids. The term “tissue” refers to a mass of connectedcells (e.g., central nervous system (CNS) tissue, neural tissue, or eyetissue) derived from a human or other animal and includes the connectingmaterial and the liquid material in association with the cells. The term“sample” also includes media containing isolated cells. One skilled inthe art may determine the quantity of sample required to obtain areaction by standard laboratory techniques. The optimal quantity ofsample may be determined by serial dilution. The term “biologicalcomponent” includes, but is not limited to nucleus, cytoplasm, membrane,epithelium, and nucleolus and stromal. The term “medical diagnosis”includes analysis and interpretation of the state of tissue material.

As will be described in detail below, digital images 110 representingbiological samples including cells, tissue samples, etc may be analyzedto provide a determination of certain known medical conditions forhumans and animals. For example, digital images 110 may be used todetermine cell proliferate disorders such as cancers, etc. in humans andanimals. Digital images 110 may be captured by, e.g., cameras 108provided in optical microscopes, where the digital images 110 representthe images seen by a human eye through the microscope. The images maythen be stored in the one or more databases 112.

FIG. 2 is a high level operation flow diagram illustrating a method 200for automated biological sample analysis. At 202, a digital image of abiological sample is received. The digital image 110 may be retrievedfrom the one or more databases 112 within the system 100. At 204, atleast one object of interest is located in the digital image. As will bedescribed below, an automated process to determine features of thebiological sample represented by the digital image 110. At 206, thelocated biological objects of interest are identified and classified todetermine a medical diagnosis or medical conclusion.

FIG. 3 illustrates an operational flow diagram that illustrates a method300 for the determination of features of the biological samplerepresented by digital images and grading such features. The method 300may be implemented in digital pathology systems to use consistentacquisition parameters to capture digital images of clinical samples.High throughput systems are presently capable of handling hospitalsentire workflows. Single user devices have been described by theseinventors as a means to bring the technology to a pathologist's desk.The method 300 may supplement the estimation and qualitative reasoningperformed by the pathologist with quantitative analysis. Thus, tocalibrate the algorithms of the method described below, pathologistshave been observed, tested and interviewed and have provided feedback onthe performance, function, utility and design of the tool.

At 302, a primary diagnosis may be performed. A pathologist may identifythe primary diagnosis using ancillary and addendum notes from thepatient's medical record. The patient information may be ascertained byreading, e.g., a barcode associate with a physical patient file. Thepatient information may indicate that the primary site is a breast. Thismay initiate a specific breast cancer algorithm from among the cancergrade library stored in, e.g., the one or more databases 112.

At 304, a tissue of interest may be identified to perform the initialregion classification. For example, the pathologist may visuallydetermine the tissue of interest to locate tumor cells, nodules,calcifications, necrosis, inflammation, and a number of identifiablecharacteristics specific to that disease site. The algorithm has beentrained by pathologists to locate the same regions of interest. Inparticular, the present disclosure leverages pattern recognitioncapabilities to precisely identify tumor cells. This providesinvestigators an opportunity to know the precise number of tumor cellsin a given sample, the area in 2D, the percentage versus other tissues.This has proven to be an extremely valuable and reliable tool forclinicians and researchers who routinely use this information to makethe best possible decisions.

The initial region classification at 304 may include one or more of thefollowing sub-processes, as illustrated in the operational flow of FIG.3B. It is noted that the present disclosure is not limited to any rangesprovided, rather they are merely provided as examples:

I. Hematoxylin and Eosin (H&E) Nucleus Identification (320):

A: Segmentation by RGB values (B<125DR)

B: Select for area (15<NUC<40 μm²) and roundness (0.6<NUC<1.0)

C: Segment adjacent nuclei by size criteria above in combination withwaterfall threshold.

D: Re-reject noncompliant structures

-   -   1. Density Dependence        -   a: Calculate nuc centroid to centroid shortest distance        -   b: Identify regional density>tissue of interest hyperplasia        -   c: Threshold density for tumor of interest            -   i. Tumor Region Classification                -   aa: Compare tissue of interest normal library of nuc                    features and density with features and density of                    the AOIs identified and reclassify accordingly.                -   ab: Optional QC checkpoint

II. Eosin Cytoplasm Identification (322):

A: Segmentation by RGB values (R>150DR)

B: Collect area, nuc number, intensity, granularity and other featuredata.

-   -   1. Nuclear:Cytoplasmic ratio (N:C) Ratio        -   a: Grow nuclei into cytoplasm to create super level cell            with constraints        -   b: Cleanup non-nucleated ROIs        -   c: Quantify N:C area measure            -   i. Non-Tumor Abnormal Classification—Reclassify any                features not identified as normal or tumor via tissue                specific library features (inflammation, reactive                stroma, fibroblasts et cetera)                III. Other Classification—Identify Regions of Interest                (ROIs) without normal or tumor classification (324):

A. Pattern Recognition Tie-In Gates

-   -   1: Clean up non-interest ROIs    -   2: Reconfirm classified ROIs    -   3: Second cleanup and segmentation review    -   4: Optional QC checkpoint        -   a. Other Classification—Classify outliers            IV. Checkerboard or multispectral approach (326).            V. Shape-based approach to cellular identification to            discern the nuclei in order to count the nuclei (328). This            may be performed in addition to, or optionally instead of,            an intensity/color approach to identify the cells at 322.

In accordance with the initial region classification process(es), breastepithelial cells can be identified as normal, hyperplasia, atypicalhyperplasia, ductal carcinoma in situ, lobular carcinoma in situ,invasive ductal carcinoma, tubular carcinoma, medullary carcinoma,mucinous carcinoma, metaplastic carcinoma, invasive criboform carcinoma,invasive papillary carcinoma, invasive micropapillary carcinoma,inflammatory breast cancer, Paget's disease of the nipple, or phyllodestumor. These are the sixteen diagnostic presentations which may beidentified. Any tissue regions which are not identified by one of thesesubtypes are classified as unidentifiable and are specificallyhighlighted for pathologist review. For simplicity of description,invasive ductal carcinoma (IDC) will be assumed as the diagnosis (80% ofall invasive breast cancers). The step for grading cancer is not whichsubtype is present, but what the grade of that subtype is presenting.

At 306, the tumor region is surveyed to assess nuclear pleomorphism.This is cancer cell classification within the method 300. The tumorcells are segmented within the (IDC) region of interest by firstidentifying the nuclei using additionally trained nuclear parametersincluding but not limited to: size (25 μm2<IDC<60 μm2); hematoxylincounterstain (160DR<layer3<210DR); shape (0.65<R<1.0; 0.45>L>1.0);texture (0.7<Haralick<0.9); nucleus to cytoplasmic ratio (0.8<N:C<4).Each nucleus generates as few as four statistics (area, intensity,roundness, texture), but as many as 210. These features are used tocreate a nucleus feature list, and the mean, median, mode, standarddeviation and significant outlier statistics area generated. Thesevalues are compared to the same features in the IDC of the breastlibrary for normal nuclei. The abnormal values are compared to thenormal range, and to itself to determine the variability in nuclearfeatures present. A nuclear pleomorphism score is generated to rangebetween 1 and 100 in one-tenth increments. Samples with more pleomorphiccells have a higher score. The companion diagnostic pleomorphic score isreduced back to the current gold standard in tertiles.

Cancer cell classification may be performed as follows:

1: Create new level for cancer cell analysis

2: Pull nuclear and cyto features into cell level

-   -   A. Determine Nuclear Waterfall        -   1: Load nuclear identification and feature data        -   2: Create distribution plot data for 16 identified nuclear            features            -   a. Score Pleomorphism—Score distribution plot against                tissue of interest library                -   i. Score I: Nuc Pleo—Rate nuc pleo into tertiles for                    tissue of interest    -   B. Mitotic Density        -   1: Identify mitotic figures (length, density, L:W ratio et            cetera)        -   2: Chessboard WSI into 400×HP fields            -   a. Counts per Field—Count identified mitosis per HP                field                -   i. Score II: Mitotic Rate—Compare number identified                    in 10 highest density HP fields and rank into known                    tertiles    -   C. Regions Fractals        -   1: Identify library of fractal analysis of structure for            tissue of interest        -   2: Run fractal dependent analysis over WSI            -   a. Score and Bin Acinar Structure—Bin tertiles of                fractal results                -   i. Score III: Tubule Formation—Score results in                    tertiles against library of tissue of interest

At 308, the cancer cell classification test and validation steps areused to correctly segment and identify tumor cells, and compare thephysical traits of those cells with the normal standard. For example,the most complex scoring algorithm is the Nottingham Breast CancerScore. This score incorporates three criteria: nuclear pleomorphic,tubule formation, and mitotic count, as opposed to the Gleason scorewhich measures patterns of anaplasia (similar to tubule formationmentioned above) or simple grades which quantify how different cancercells look compared to normal cells of the same type.

Each of the Nottingham criteria is given a score between one and three(higher scores are more perturbed). The sum of these scores (3-9) isthen re-segmented into low grade (I=3-5), moderate grade (II=6-7) andhigh grade (III=8-9) Nottingham Grades.

Thus, by using an extensive database of well over 15,000 tissue samplesand complex histology, pattern recognition software and algorithms havebeen designed and optimized to recognize tumor regions within histologysamples, segment the individual nuclei, and report the tumor burden asthe number of tumor cells, the area of tumor versus other cell types,and the percentage of tumor cells. To achieve accuracy and reliabilityof each algorithm, exhaustive tests and validation steps are used tocorrectly identify and segment tumor cells, and compare the physicaltraits of those cells with the normal standard. Each value is reportedwith a specificity and sensitivity metric to be used to ensure thehighest quality analysis for heterogeneous sample qualities.

FIGS. 4A and 4B illustrate several representative graded digital imagesof biological samples. In FIG. 4A, image A is a grade I invasive breastcancer; image B is a grade III invasive breast cancer; image C is abinary masked evaluation of acinar formation grade I; image D is abinary masked evaluation of acinar formation grade I; image E is a gradeII nuclear pleomorphism; image F is a nuclear identification forevaluation of nuclear size, roundness and nucleus: cytoplasm ratio;image G is a grade III mitotic count; and image H is a mitotic figureidentification and evaluation of grade. The method 300 mimics the aboveby first identifying the tumor region, segmenting the cells andaddressing the properties of nuclear features, tubule formation and theexact number of mitotic counts per 10 (40×) high powered fields (asdescribed by the AJCC guideline).

In FIG. 4B, image A illustrates how pathologists select the regions ofinterest with a histology pen by drawing directly on the glass slide.However, even tumor regions of tissues are made up of a complex networkof heterogeneous cell types including, but not limited to, stroma,fibrosis, inflammation, vasculature, blood cells, bone and cartilage,adipocytes or fat cells, muscle et cetera. When a tissue is selected fordownstream analysis, each of these cell types plays its own importantrole in the outcome and subsequent data. With the advent of histologicalpattern recognition, a quantitative tool has been developed to identifyindividual cancer cells from the rest of the tissue cell populationswith precision. Image B depicts an H&E whole slide image of colon cancerwith millimeter resolution. Image C is a green mask of epithelial tumorand yellow mask identifying other tissues. Images D and E depict thesame sample at micron levels of resolution. Each cell may be countedindividually to establish a precise tumor burden.

The tumor region is surveyed by the pathologist and 10 high power fields(40×) are reviewed in regions estimated to include the highest rate ofmitosis, and the number of mitotic events is counted. 0-9 mitoses score1; 10-19 mitoses score 2; 110 or more score 3.

In accordance with some implementations, the system 100 has the entireIDC tumor available to survey, and the area of one high power field(hpf) is known. The area of the hfp depends on the scanning instrumentused. In this example the Aperio ScanScope XT will be assumed (0.37μm2/pixel) and the field is tiled into hpf. Each field is surveyed formitotic figures with the features including but not limited to: size (10μm2<IDC<20 μm2); hematoxylin counterstain (220DR<layer3<255DR); shape(0.00<R<0.45) (0.85>L>1.0); with a defined mitotic figure within 4 μm.The count is calculated for each pair in each hfp and the fields withthe largest mitotic counts are identified and the figures are summed.The results are compared with the grade system standard as described forscores 1-3 above.

The tubule formation is estimated by the pathologist as either themajority of the tumor (>75%), a moderate degree (10-75%) or little tonone remaining (<10%). The companion diagnostic method 300 determinestubule formation by calculating the larger structure of the tumor insurface area box counting fractal dimensions to calculate and record thecomplexity of the objects. The system 100 may also determine theneighboring objects for each cell, which can be described as near (≦2μm2) or distant >2 μm2) and the direction of distance. This is used tocreate the gland formulation score which is based on the premise thatcells with two neighbors on opposite sides, but not on the other sides,are likely cells within glands, whereas cells with more than two nearcells, or no near cells, are likely part of a less organized structure.The fractal dimension and gland formulation score are combined todetermine the tubule formation. IDC cells are then calculated as a wholeto fit back into the grading criteria described for the pathologistabove.

The pathologist tallies each of the criteria above to calculate a sumNottingham score. The system 100 may do the same, and draft a simplereport (at 310) for each of the three sub-scores and the ultimateNottingham Grade of I, II or III. The raw data are retained as ancillarydata to be reviewed at a later date as needed. The electronic record isimprinted as a layer with the digital slide, and can be linked to theLaboratory Information Management System and populated with the patienthistory, images, and other ancillary data.

In some implementations, the system 100 may apply the primary tumorsite's accepted grade(s) to each slide in patient case, and compares theresults to the library of known normal descriptions for that tumormorphology. The comparison score is then binned, e.g., into the samescoring mechanism accepted for use by the AJCC and CAP guidelines andthe quantitative score is recorded in a simple digital record sheet. Inthis case the algorithms run sequentially until all libraries for theprimary site are exhausted and recorded.

Thus, as described above, the method 300 works seamlessly with digitalwhole slide scans from either instrument throughput type to create asimple, easy to use score which pathologists may then use as a companiondiagnostic tool. As such, pathologists will have a simple, easy to use,companion diagnostic to enhance reliability, repeatability, accuracy andquantification is paramount. Similarly, pathologists will be providedwith a tool which can insure a more accurate diagnosis. Furthermore, thepresent disclosure lays the groundwork for grading criteria, featureanalysis and biological progression of disease indicators which willultimately increase the precision of pathologist's evaluations and willmore carefully examine the needs of individual patients.

Now with reference to FIGS. 5A-5C, there is illustrated several views ofthe desktop slide scanner 500 of the present disclosure. As shown inFIGS. 5A-5B, the desktop slide scanner 500 (platform 502) includes aframe 504, which is the structure of the desktop slide scanner 500, andan imaging housing 520, which enables detection of the sample atsufficient magnification and resolution and enables functionality of thespecimen location motors, and the handheld device itself.

The frame 504 is a desktop base device having dimensions ofapproximately 6″ W×11″ L×3″ H, or other dimensions such that it suitablyoperates within a desktop environment. The frame 504 may be made from asuitably lightweight rigid material, such as aluminum, plastic, etc.,and supported by four legs 506 at each corner of similar materials. Thelegs 506 may be, e.g., 2″ in height. On the top of the frame 504 is apersonal communication device seat 510, having dimensions suitable toreceive a handheld or portable device, such as a smart phone, tabletdevice, etc. The seat 510 receives the portable device to a light adetector (e.g., a camera) embedded within the particular portable devicewith a detector eye 508. Integration may be achieved withhandheld/portable devices, such as iPhones, iPads, Android devices,BlackBerrys, etc. or any tablet-style computer. As will be describedbelow, using the built-in detector (e.g., camera 108) ofhandheld/portable devices, digital images may be taken of slides loadedinto the desktop slide scanner 500. The camera 108 may be a CCD (ChargeCoupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). Theaforementioned may be achieved through a combination of hardware andsoftware, as described below. Optional device clips 512 may be providedon each side of the seat 510 to secure a handheld/portable device to theframe 504.

Within the device seat 510 there is provided three openings. A firstopening is located at the camera detector eye 508. A hole may be fittedwith a rubber seal formed specific to the dimensions of the CCD/CMOSdetector of the handheld device received by the seat 510. The sealexcludes light from entering the imaging field during use. Second andthird openings are designed to allow pass-through of two cords, one fora network connection (see, FIG. 6) and the second for a power connection516. Where the connection cords are considered semi-permanent (i.e., arenot removed after each use), these can be outfitted with spring loadedcord seats in order to make connecting the device to the cords aneffortless operation. In some implementations, power cords may not benecessary if, e.g., powerpad technology is available to fit inseamlessly within the seat 510. Still further, the wired networkconnection may be alternatively replaced by a wireless connection, suchas, Wi-Fi, 3G, 4G, etc.

With reference to FIGS. 5B-5C, the imaging housing 520 may be formed asa light-tight enclosure with approximately the same lateral dimensionsof the frame 504. The imaging housing 520 serves to move a biologicalsample disposed on a slide into the imaging field, where the sample maybe magnified to be detected. For example the slide may be a standardglass microscopy slide sample (20 mm×50 mm×2 mm). In order to move theslide, the imaging housing 520 is outfitted with a laterally slidingtray shown as a slide holder 522. The slide is placed into, e.g., a 1 mmrecessed 20 mm×50 mm tray 524 to hold the sample in a known, stationaryposition. The slide tray 524 may be made from aluminum or otherlightweight material, and is mounted under the frame 504, and within theimaging housing 520. The slide tray 524 may be mounted on track threadedspiral guides 526 moving the length of the x-direction (e.g., 11″) andmounted in the y-direction (6″) axis by two mounting clips on each axis.The mounting clips may be made from aluminum or other lightweightmaterial. The tray 524 may be outfitted with a third, z-axis step motor528 which moves in, e.g., 0.2±0.01 μm increments. It is noted that anydrive mechanism may be used to move the tray 524, such asrack-and-pinions, belts, chains, linear motors, ball screws, and fluidpressure cylinders.

Magnification may be achieved through an imaging portal 514. The imagingportal 514 may include a low-profile super optical telecentric lens with20× magnification, 0.35 numerical aperture, and a 36.5 mm workingdistance. The lens may be mounted on the underside of the imaging portal514. In some implementations, the lens may hinge on its own motor, whichmay be instructed by software to move the lens in and out of place asimage acquisition is initiated. A light source may be provided withinthe imaging housing 520, for example, a light-emitting diode or a xenonlamp. Light from the light source may be reflected on a mirror, ordirectly illuminated, on the sample disposed on the slide of interest.

FIG. 6 is a block diagram illustrating another exemplary digital imageacquisition system 600. Moreover, like reference numerals designatecorresponding components, as described above and will not be describedagain. The system 600 may include a desktop slide scanner 500 thatincorporates the camera 108. As will be appreciated, the camera 108 maybe stand-alone camera or part of handheld/portable device as describedabove in FIGS. 5A-5C. The desktop slide scanner 500 is adapted tocapture digital images of slides 110 in cooperation with the camera 108of, e.g., a handheld/portable device. Alternatively or additionally, thedesktop slide scanner 500 may be adapted to capture digital images underthe command and control of the one or more computers 102 connected overthe one or more communications networks 114.

Operation of the Desktop Slide Scanner 500

FIG. 7 is a flow diagram illustrating a process for obtaining a digitalimage from a slide having a biological sample disposed thereon.Generally, the operation includes one or more of the operations of slideloading, tissue identification, sample focusing, digital imaging, datatransfer for the purposes of data management, and analysis. In someimplementations, the function may be driven by the handheld/portabledevice 108 itself. For example, drivers scripted for the iOS (Apple)operating system and Android applications may communicate with a scanbutton provide on the desktop slide scanner 500 and the drive motorsthereof. Alternatively or additionally, the one or more computers 102may operate the desktop slide scanner 500 by communication with thehandheld/portable device or the desktop slide scanner 500 itself.Alternatively or additionally, operation of the desktop slide scanner500 may be initiated by a touch of a display or button of thehandheld/portable device or actuation of a graphical element on thedisplay 104.

At 702, a user places a handheld/portable device on the seat. The usermay place the device 108 on the seat 510 and allow spring loaded cordinjectors to plug-in to the device. This action may which turn thedevice 108 on. At 704, the scanning application is launched. This mayoccur automatically upon seating the device 108 or manually by useraction. At 706, the user loads a slide into the tray seat and actuates ascan button. This initiates the scan process at 708. During the scanprocess, the tray 522 is moved to position the slide such that a centerof the image portal 514 is over a calibration area and a backgroundimage may then be captured. The lens position is checked and moved intoposition distant from the image portal.

At 710, images of the slide are acquired. Here, the slide 110 is thenmoved such that the top left of the slide 110 is placed under the imageportal 514. Images are taken in tile formation from left to right andtop to bottom to cover the entire slide 110. Alternatively, an array ofdetector chips may be provided, or line scanning detection performed toacquire the images. At 712, the resulting stitched image is thenprocessed through a ‘find tissue’ algorithm within the deviceapplication. The find tissue algorithm highlights objects with morecontrast than the calibration image in addition to an area larger thanan area determined by the user or default setting. Coordinates of theseregions are auto-targeted for scanning. A checkerboard coordinate map isestablished in the software with the known pixel size dimensions for thedevice used. Where tissue is not found, the user may optionally manuallyselect scan regions. Next, the lens position is moved into the imageportal window. From left to right and top to bottom each coordinate onthe map is imaged at, e.g., 20× magnification or other suitablemagnification to create magnified images of the detected tissue regionsof the slide 110.

At 714, the slide is ejected. A new slide may be inserted where theprocess returns to 706 or the process may end. Each image may be cachedto the database 112 over the network 114. In some implementations, theimage may be cached to the device 108 or computer 102. The device 108 orcomputer 102 may display the information on the screen or display 104,and the low magnification pictures may be saved for non-network databasereview and algorithm queuing.

Thus, as described above, there is a simple, easy to use, highresolution whole slide imaging device for individual use. The smallfootprint is ideal for single use. The device integration is efficient,and takes advantage of the optimized imaging, computing and softwarescripting efforts provided by Apple and Android. The one touch ease ofuse is ideal to meet the needs of pathologists without the desire to usecomplex instrumentation. The optical lens is capable of matchingmagnification, NA and WD in a confined space. Furthermore, this lens iscapable of utilizing a myriad of detectors for numerous devices.

Further, the present disclosure describes a system and methods thataddress a number of current challenges including, but not limited to: 1)losing or damaging glass slides; 2) subjective or inadequatedocumentation of specific regions with spatially limited still framedigital imaging; 3) inability to document and manage one's own cases; 4)inefficient sharing between colleagues; 5) reliance on techniciansunfamiliar with your personal workflow. In addition it makes several newopportunities available including 1) quantifiable analysis by FDAapproved algorithms; 2) documentation for tumor board, departmentalmeetings or other sharing opportunities; 3) single button ease of useand others.

FIG. 8 shows an exemplary computing environment in which exampleimplementations and aspects may be implemented. The computing systemenvironment is only one example of a suitable computing environment andis not intended to suggest any limitation as to the scope of use orfunctionality.

Numerous other general purpose or special purpose computing systemenvironments or configurations may be used. Examples of well knowncomputing systems, environments, and/or configurations that may besuitable for use include, but are not limited to, personal computers(PCs), server computers, handheld or laptop devices, multiprocessorsystems, microprocessor-based systems, network PCs, minicomputers,mainframe computers, embedded systems, distributed computingenvironments that include any of the above systems or devices, and thelike.

Computer-executable instructions, such as program modules, beingexecuted by a computer may be used. Generally, program modules includeroutines, programs, objects, components, data structures, etc. thatperform particular tasks or implement particular abstract data types.Distributed computing environments may be used where tasks are performedby remote processing devices that are linked through a communicationsnetwork or other data transmission medium. In a distributed computingenvironment, program modules and other data may be located in both localand remote computer storage media including memory storage devices.

With reference to FIG. 8, an exemplary system for implementing aspectsdescribed herein includes a computing device, such as computing device800. In its most basic configuration, computing device 800 typicallyincludes at least one processing unit 802 and memory 804. Depending onthe exact configuration and type of computing device, memory 804 may bevolatile (such as random access memory (RAM)), non-volatile (such asread-only memory (ROM), flash memory, etc.), or some combination of thetwo. This most basic configuration is illustrated in FIG. 8 by dashedline 806.

Computing device 800 may have additional features/functionality. Forexample, computing device 800 may include additional storage (removableand/or non-removable) including, but not limited to, magnetic or opticaldisks or tape. Such additional storage is illustrated in FIG. 8 byremovable storage 808 and non-removable storage 810.

Computing device 800 typically includes a variety of computer readablemedia. Computer readable media can be any available media that can beaccessed by device 800 and include both volatile and non-volatile media,and removable and non-removable media.

Computer storage media include volatile and non-volatile, and removableand non-removable media implemented in any method or technology forstorage of information such as computer readable instructions, datastructures, program modules or other data. Memory 804, removable storage808, and non-removable storage 810 are all examples of computer storagemedia. Computer storage media include, but are not limited to, RAM, ROM,electrically erasable program read-only memory (EEPROM), flash memory orother memory technology, CD-ROM, digital versatile disks (DVD) or otheroptical storage, magnetic cassettes, magnetic tape, magnetic diskstorage or other magnetic storage devices, or any other medium which canbe used to store the desired information and which can be accessed bycomputing device 800. Any such computer storage media may be part ofcomputing device 800.

Computing device 800 may contain communications connection(s) 812 thatallow the device to communicate with other devices. Computing device 800may also have input device(s) 814 such as a keyboard, mouse, pen, voiceinput device, touch input device, etc. Output device(s) 816 such as adisplay, speakers, printer, etc. may also be included. All these devicesare well known in the art and need not be discussed at length here.

It should be understood that the various techniques described herein maybe implemented in connection with hardware or software or, whereappropriate, with a combination of both. Thus, the processes andapparatus of the presently disclosed subject matter, or certain aspectsor portions thereof, may take the form of program code (i.e.,instructions) embodied in tangible media, such as floppy diskettes,CD-ROMs, hard drives, or any other machine-readable storage mediumwhere, when the program code is loaded into and executed by a machine,such as a computer, the machine becomes an apparatus for practicing thepresently disclosed subject matter.

Although exemplary implementations may refer to utilizing aspects of thepresently disclosed subject matter in the context of one or morestand-alone computer systems, the subject matter is not so limited, butrather may be implemented in connection with any computing environment,such as a network or distributed computing environment. Still further,aspects of the presently disclosed subject matter may be implemented inor across a plurality of processing chips or devices, and storage maysimilarly be affected across a plurality of devices. Such devices mightinclude PCs, network servers, and handheld devices, for example.

Although the subject matter has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the specific features or acts described above.Rather, the specific features and acts described above are disclosed asexample forms of implementing the claims.

What is claimed:
 1. A computer-implemented method for determining andgrading of features of a biological sample represented by a digitalimage, comprising: performing an initial region classification toclassify cells within the biological sample; surveying a tumor region toassess disease state to perform a cancer cell classification; gradingthe cancer cell classification of the biological sample; and generatinga report of the graded biological sample.
 2. The method of claim 1,performing the initial region classification further comprising:applying a pattern recognition algorithm to the digital image toidentify tumor cells.
 3. The method of claim 2, further comprising:determining a number of tumor cells in the biological sample;determining an 2D area of the tumor cells; and determining a ratio oftumor cells to non-tumor cells in the biological sample.
 4. The methodof claim 1, performing the initial region classification furthercomprising performing one of a Hematoxylin and Eosin (H&E) nucleusidentification, an Eosin cytoplasm identification, a multispectralanalysis and a shaped-based analysis.
 5. The method of claim 4, whereinthe H&E nucleus identification comprises segmenting a nucleus byred-green-blue (RGB) values and selecting the nucleus in accordance withpredetermined area and roundness criteria.
 6. The method of claim 4,wherein the Eosin cytoplasm identification comprises segmenting anucleus by red-green-blue (RGB) values and determining anuclear:cytoplasmic (N:C) ratio.
 7. The method of claim 4, furthercomprising performing an quality control checkpoint operation toclassify outliers.
 8. The method of claim 1, further comprising:classifying breast epithelial cells into a predetermined category; andproviding a notification that the biological sample has not beenclassified by the initial region classification.
 9. The method of claim1, surveying the tumor region further comprising assessing nuclearpleomorphism.
 10. The method of claim 9, further comprising determininga nuclear parameter, the nuclear parameter being at least one of size,hematoxylin counterstain, shape, texture and nucleus to cytoplasmicratio.
 11. The method of claim 10, wherein the size is between 25 μm2and 60 μm2, wherein the hematoxylin counterstain is between 160DR and210DR, wherein the shape has a radius between 0.65 and 1.0 and a lengthbetween 0.45 and 1.0, wherein the texture Haralick value is between 0.7and 0.9, and wherein the nucleus to cytoplasmic ratio is between 0.8 and4.
 12. The method of claim 1, grading the cancer cell furthercomprising: determining a nuclear waterfall of the cancer cell;determining a mitotic density; and determining region fractals.
 13. Themethod of claim 12, wherein determining a nuclear waterfall comprises:loading nuclear identification and feature data; and creatingdistribution plot data for predetermined nuclear features.
 14. Themethod of claim 13, wherein determining the mitotic density comprisesscoring the distribution plot against a tissue of interest library. 15.The method of claim 14, wherein determining regions fractals comprises:identifying a library of fractal analysis of structure for the tissue ofinterest; running a fractal dependent analysis.
 16. The method of claim1, further comprising comparing the cancer cell with a standard scoringalgorithm.
 17. The method of claim 16, wherein the standard scoringalgorithm is the Nottingham Breast Cancer Score.
 18. A desktop slidescanner, comprising: a frame that defines a seat into which a portabledevice is disposed and a detector eye that aligns an embedded imagingdevice of the portable device with the frame; an imaging portal mountedto the frame that includes a lens having a predetermined magnification;and a light-tight enclosure that is received within the frame, thelight-tight enclosure further including a moveable a slide tray having aslide seat adapted to receive a slide, wherein the slide tray is movedwithin the light-tight enclosure and imaged by the imaging device, andwherein the images are magnified at the predetermined magnification bythe lens within the imaging portal.
 19. The desktop slide scanner ofclaim 18, further comprising a seal that forms around the imaging deviceof the portable device to exclude light from entering an imaging fieldduring use.
 20. The desktop slide scanner of claim 18, furthercomprising a network connection provided to the portable device totransfer images to a remote repository.
 21. The desktop slide scanner ofclaim 18, wherein the slide tray is moveably mounted on a track.
 22. Thedesktop slide scanner of claim 18, further comprising a step motor tomove along an axis of the desktop slide scanner in predeterminedincrements.
 23. The desktop slide scanner of claim 18, wherein theimaging portal 514 provides a predetermined level of magnification. 24.A computer-implemented method for scanning a biological sample andcreating a digital image thereof, comprising: receiving a portabledevice within a slide scanner frame to align an imaging device of theportable device with a detector eye; performing a calibration of a slideregion; imaging the slide in a tile formation; stitching images of theslide together; detecting tissue disposed on the slide; scanning regionsof the slide where tissue is detected at a predetermined magnification;and saving images of the scanned regions to a repository.
 25. The methodof claim 24, further comprising: launching a scanning application uponreceiving the portable device within the slide scanner; and initiating ascan process to perform the calibration of the slide region.
 26. Themethod of claim 25, further comprising: moving a tray holding the slideto position the slide such that a center of an image portal is over acalibration area; capturing a background image may then be captured. 27.The method of claim 24, further comprising highlighting objects withmore contrast than the calibration an image of the slide region.
 28. Themethod of claim 27, further comprising auto-targeted the object forscanning.
 29. The method of claim 24, further comprising saving lowmagnification pictures may be saved for non-network database review andalgorithm queuing.